A Snapshot Utility for a Distributed Object-Oriented Database System

Thor is an object-oriented database management system that is designed to be scalable in a distributed environment. Thor allows objects to be stored persistently and accessed efficiently while providing ACID transaction semantics. In this paper, we propose to extend Thor with a snapshot utility. Snapshots are useful for a wide range of applications, including data analysis, data mining and database recovery. In order to provide high performance in Thor, it is essential that the snapshot utility be efficient in both the archiving and the retrieval of snapshot so that interference with concurrent transactions in the system is minimized. We propose a lazy gossip-based approach to taking snapshots efficiently. In this approach snapshots are propagated throughout the system via a lazy pairwise exchange of gossip messages between servers. The propagation of messages is either done in the background or piggybacked on other communication messages so that they are non-intrusive to concurrent transactions. In addition, we propose to create snapshots in an incremental and lazy manner to further improve the efficiency and to minimize the space requirements of storing the snapshots. Our approach is novel in that it makes use of the server’s write buffer to allow the lazy propagation and creation of snapshots. In our system, the data pages of a snapshot are automatically archived after they have been created. These pages are later retrieved on-demand when the user runs a transaction on a snapshot of the database. The archival and retrieval of snapshots is completely transparent to the user. The retrieved data pages of snapshots are installed into the client’s persistent cache. We extend Thor’s cache management mechanism to support the caching of multiple versions of the database at the same time, and also to allow compaction of the database across these versions without cache contamination. Finally, we propose a peer-to-peer infrastructure for the underlying storage facility required for archiving the snapshots. Peer-to-peer systems have the desirable qualities of high availability, fault-tolerance, automatic loadbalancing and self-configuration. More importantly, the cooperative storage paradigm in peer-to-peer systems allows us to utilize the otherwise unused storage in the participating storage servers. We extend the DHash layer in the Cooperative File System for archiving data pages of snapshots by providing persistent storage and adding a self-repair mechanism to improve data survivability.